Documentation

inbox/ADR-036 Remediation Plan.md

ADR-036 Remediation Plan

This document outlines the plan to bring the codebase into compliance with ADR-036: Database Project Separation.

Executive Summary

ADR-036 mandates:

  1. .Abstractions → only reference other .Abstractions (no EF, no .Database)
  2. .Database → only reference foundational .Database projects (Prism, Identity) for FK modeling
  3. Services → use .ApiClient (Kiota) for cross-service communication, not direct .Database access

Current state has significant violations across BBU, IoT, Identity, Workflow, Printing, Spatial, and Catalog components.

Violation Inventory

Category A: .Abstractions → .Database Violations

Component References Types Used Complexity
Printing.Abstractions Printing.Database PrintJob, PrintJobData (EF entities in response DTOs) M
Spatial.Abstractions Spatial.Database Address entity, SpatialDb in ImportAddressRequest S-M
Bbu.Abstractions Catalog.Database, Spatial.Database Item, LocationCategory, Movement in BasketDetailsResponse M
Iot.Abstractions CoreData.Database, Iot.Database, Identity.Database Full DB write logic in processor base classes L-XL
Identity.Abstractions CoreData.Database, Prism.Database, Identity.Database Language, UserSecurity, TenantHelper with DB queries S-M
Workflow.Abstractions Catalog.Database Catalog EF entities in workflow definitions M

Category B: Service → Foreign .Database Violations

Component Foreign DB References Usage Complexity
BBU Catalog, Iot, SystemEnvironment, Transport, Spatial, Prism, Identity Full cross-DB orchestration, reads AND writes XL
Spatial Identity.Database Auth config, tenant scoping (may be acceptable) S

Category C: .Database → Non-Foundational .Database Violations

Component References Purpose Complexity
Bbu.Database Catalog.Database, Spatial.Database FK relationships to items/locations S-M*
Workflow.Database Catalog.Database, CoreData.Database, Spatial.Database Workflow rules over external entities S-M*
Iot.Database CoreData.Database, Spatial.Database, SystemEnvironment.Database Device reference data FKs S-M*
Catalog.Database CoreData.Database, FileHandling.Database, SystemEnvironment.Database Lifecycle, files, organizations S-M*
SystemEnvironment.Database CoreData.Database Vocab terms reference quantities/units S*

* Complexity reduced by leveraging Passport pattern - FK to prism.passports instead of shadow entities

Remediation Phases

Phase 0: Guardrails (1-2 days)

Goal: Prevent new violations while fixing existing ones.

Tasks:

  1. Add Roslyn analyzer or CI check that flags:
    • Any .Abstractions project referencing .Database
    • Any service project referencing foreign .Database (except own)
    • Any .Database referencing non-foundational .Database
  2. Create [ADR036Exception] attribute for temporary suppressions with required justification
  3. Document exceptions in this file

Phase 1: Simple Cleanups (1-2 days)

Goal: Quick wins with minimal risk.

Task Component Action
1.1 Spatial.Abstractions Remove stray using Acsis.Dynaplex.Engines.Spatial.Database; from LocationImportRecord.cs
1.2 Identity.Abstractions Change UserToken.CurrentCulture/CurrentLanguage from CoreData.Database.Language to CoreData.Abstractions.Primitives.Language
1.3 All .Abstractions Audit for any other unused .Database usings and remove

Phase 2: DTO Replacements in Abstractions (1-2 weeks)

Goal: Replace EF entities exposed in Abstractions with proper DTOs.

2.1 Printing.Abstractions (M)

Current:
  PrintJobWithDetails.PrintJob → Printing.Database.PrintJob
  PrintJobDataWithFieldName : Printing.Database.PrintJobData

Fix:
  1. Create PrintJobDto, PrintJobDataDto in Printing.Abstractions
  2. Update PrintJobWithDetails to use PrintJobDto
  3. Update PrintJobDataWithFieldName to inherit from PrintJobDataDto
  4. Add mapping in Printing service: PrintJob → PrintJobDto
  5. Remove Printing.Database reference from Printing.Abstractions.csproj

2.2 Bbu.Abstractions (M)

Current:
  BasketDetailsResponse uses:
    - Catalog.Database.Item
    - Spatial.Database.LocationCategory
    - Spatial.Database.Movement
  Extensions.HasNecessaryParameters targets Catalog.Database.Item

Fix:
  1. Create BBU-specific DTOs:
     - BasketItemSummary
     - BasketLocationSummary
     - BasketMovementSummary
  2. Update BasketDetailsResponse to use these DTOs
  3. Move Extensions.HasNecessaryParameters to BBU service (target EF entity there)
  4. Add mapping in BbuDataService: EF entities → DTOs
  5. Remove Catalog.Database, Spatial.Database from Bbu.Abstractions.csproj

2.3 Identity.Abstractions (S-M)

Current:
  AuthResponse.UserLogonProfile → Identity.Database.UserSecurity
  TenantHelper.GetDefaultTenantIdAsync uses IdentityDb directly

Fix:
  1. Create UserSecurityDto in Identity.Abstractions
  2. Update AuthResponse to use UserSecurityDto
  3. Move TenantHelper to Identity service, expose via API:
     - New endpoint: GET /api/tenants/default
     - Generate Kiota client method
  4. Replace external TenantHelper usages with Identity.ApiClient
  5. Remove Identity.Database, Prism.Database, CoreData.Database from Identity.Abstractions.csproj

2.4 Workflow.Abstractions (M)

Current:
  Uses Catalog.Database entities in workflow definitions

Fix:
  1. Replace Catalog EF entities with:
     - Catalog.Abstractions DTOs, OR
     - Workflow-specific DTOs with just IDs (Guid ItemTypeId, etc.)
  2. Add mapping in Workflow service
  3. Remove Catalog.Database from Workflow.Abstractions.csproj

2.5 Spatial.Abstractions - ImportAddressRequest (M)

Current:
  ImportAddressRequest.Address → Spatial.Database.Address
  ImportAddressRequest.IsAddressEqualToImportRecord uses SpatialDb

Fix:
  1. Create AddressModel DTO in Spatial.Abstractions
  2. Move IsAddressEqualToImportRecord logic to Spatial service
  3. Remove Spatial.Database from Spatial.Abstractions.csproj

Phase 3: Passport-Based FK Simplification (3-5 days)

Goal: Remove non-foundational .Database → .Database references by leveraging the Passport pattern.

Key Insight: Passport Owns All Primary Keys

Most entities in the system have their Id as a FK to prism.passports.global_id. Since Prism.Database is an allowed foundational dependency, we can:

  1. Remove project references to non-foundational .Database projects
  2. FK directly to Passport instead of the specific entity table
  3. Use PlatformTypeId for type-safety validation at the application layer if needed

The Pattern

Before (Violation):

// Bbu.Database references Catalog.Database
using Acsis.Dynaplex.Engines.Catalog.Database;

public class TagReadProcessing
{
    public Guid ItemId { get; set; }
    public Item Item { get; set; }  // Navigation to Catalog entity
}

// In BbuDb.OnModelCreating:
modelBuilder.Entity<TagReadProcessing>()
    .HasOne(x => x.Item)
    .WithMany()
    .HasForeignKey(x => x.ItemId);

After (Passport-Based):

// Bbu.Database only references Prism.Database (allowed)
using Acsis.Dynaplex.Engines.Prism.Database;

public class TagReadProcessing
{
    public Guid ItemId { get; set; }
    // No navigation property - use ApiClient to fetch Item data
}

// In BbuDb.OnModelCreating:
modelBuilder.Entity<TagReadProcessing>(b => {
    // FK to Passport (which owns the ID)
    b.HasOne<Passport>()
        .WithMany()
        .HasForeignKey(x => x.ItemId)
        .HasConstraintName("fk__bbu__tag_read_processing__prism__passports__global_id");
});

What This Achieves

Aspect Before After
Project reference Bbu.Database → Catalog.Database Bbu.Database → Prism.Database (allowed)
Database FK bbu.x.item_id → catalog.items.id bbu.x.item_id → prism.passports.global_id
Referential integrity ✅ Enforced ✅ Enforced (via Passport)
Type-specific constraint ✅ Must be an Item ⚠️ Must be any valid Passport
Business data access Navigation property ApiClient HTTP call

Type Safety Consideration

The trade-off is that the DB constraint now says "must be a valid Passport" rather than "must be an Item specifically." Options:

  1. Accept this - Application layer validates PlatformTypeId when needed
  2. Add CHECK constraint - CHECK (platform_type_id = 101) where 101 is Item's PTID
  3. Use shadow entities (fallback) - Only for non-passport-backed entities

Component Updates

3.1 Bbu.Database (S-M)
  • Remove: Catalog.Database, Spatial.Database references
  • Keep: Prism.Database (foundational)
  • Change FKs for Item, Location, Movement to point to Passport
  • Navigation properties removed; use Catalog.ApiClient, Spatial.ApiClient for data
3.2 Workflow.Database (S-M)
  • Remove: Catalog.Database, CoreData.Database, Spatial.Database references
  • Keep: Prism.Database
  • Change FKs for passport-backed entities (ItemType, Item, Movement, etc.) to Passport
  • Exception: UnitOfMeasure, Quantity - check if passport-backed; if not, use shadow entity pattern
3.3 Iot.Database (S-M)
  • Remove: CoreData.Database, Spatial.Database, SystemEnvironment.Database references
  • Keep: Prism.Database
  • Change FKs for Location, Organization to Passport
  • Exception: Quantity*, UnitOfMeasure - verify passport status
3.4 Catalog.Database (S-M)
  • Remove: CoreData.Database, FileHandling.Database, SystemEnvironment.Database references
  • Keep: Prism.Database
  • Change FKs for Organization, File to Passport
  • Note: Coordinate with ongoing Prism/Catalog work
3.5 SystemEnvironment.Database (S)
  • Remove: CoreData.Database reference
  • Keep: Prism.Database
  • Change quantity/unit FKs to Passport if passport-backed

Shadow Entity Fallback

For entities that are NOT passport-backed (lookup tables, legacy tables), use the shadow entity pattern:

// Only needed for non-passport-backed entities
namespace Acsis.Dynaplex.Engines.Bbu.Database.Shadows;

/// <summary>
/// Shadow entity for a non-passport-backed lookup table.
/// Used ONLY for FK relationships.
/// </summary>
internal class SomeLookupShadow
{
    public int Id { get; set; }
}

// In DbContext:
modelBuilder.Entity<SomeLookupShadow>(b => {
    b.ToTable("some_lookup", "other_schema", t => t.ExcludeFromMigrations());
    b.HasKey(x => x.Id);
});

Verification Step

Before removing each .Database reference, verify:

  1. Which entities from that project are actually used
  2. Whether each entity is passport-backed (has Id FK to passports.global_id)
  3. If not passport-backed, use shadow entity instead

Phase 4: IoT Abstractions Refactor (1-2 weeks)

Goal: Move DB logic out of Abstractions into service layer.

Current Problem:

  • ZebraRfidProcessor, DatalogicBlobProcessor are base classes in IoT.Abstractions
  • They contain full DB write logic using IotDb, IdentityDb
  • This forces Abstractions to depend on multiple .Database projects

Fix:

1. Keep in Iot.Abstractions (pure contracts):
   - Message format models (MQTT payloads)
   - Configuration classes (ZebraRfidProcessorConfiguration)
   - Hook interfaces (ITagReadHandler, IManagementEventHandler)

2. Move to Iot service project:
   - DB write logic
   - Tenant resolution
   - Base processor implementations that use DbContext

3. Restructure inheritance:
   - Abstract base in Abstractions defines hook points
   - Concrete implementation in Iot service provides DB access
   - BBU/other consumers implement hooks, call Iot via ApiClient

4. Remove Database references from Iot.Abstractions.csproj

Phase 5: BBU Service ApiClient Migration (3-4 weeks+)

Goal: Replace direct DB access with HTTP API calls.

Current State:
BBU service directly uses: CatalogDb, SpatialDb, TransportDb, IotDb, IdentityDb, PrismDb

Strategy:

5.1 Audit and Categorize (3-5 days)

For each cross-DB usage, determine:

  • Read-only vs Write
  • Performance-critical vs Background
  • Existing API available vs Needs new endpoint

5.2 Read-Only Migrations (1-2 weeks)

Replace direct reads with ApiClient calls where APIs exist:

Current Replacement
CatalogDb.Items.FirstOrDefault(x => x.Id == id) catalogApiClient.Items[id].GetAsync()
SpatialDb.Locations.Where(...) spatialApiClient.Locations.GetAsync(filter)
TransportDb.Shipments.Include(...) transportApiClient.Shipments[id].GetAsync()

5.3 New API Endpoints (1-2 weeks)

Create missing endpoints in owning components:

  • Catalog: Bulk operations, search enhancements
  • Spatial: Movement creation API
  • Transport: Shipment lifecycle APIs
  • Identity: Tenant resolution API (if not already done in Phase 2)

5.4 Write Migration (2-3 weeks)

Replace cross-DB writes with API calls:

  • Mark current write paths as [Obsolete]
  • Implement via ApiClient
  • Handle transactions via saga/orchestration patterns where needed
  • Consider domain events for eventual consistency

High-Risk Services to Address:

  1. BasketProcessingService - Creates Items, Movements across DBs
  2. ShipmentLifecycleProcessor - Manages Transport entities
  3. BbuInitializationService - Seeds data across all DBs
  4. UnknownDestinationProcessor - Creates shipments/allocations
  5. TagReadReplayService - Cross-schema SQL joins

Dependency Order

Phase 0 (Guardrails)
    ↓
Phase 1 (Simple cleanups)
    ↓
Phase 2.3 (Identity.Abstractions - TenantHelper → API)
    ↓ (enables other phases to use Identity.ApiClient)
Phase 2.1, 2.2, 2.4, 2.5 (Other Abstractions DTOs) [parallel]
    ↓
Phase 3 (Passport-based FK simplification) [parallel with Phase 4]
    ↓
Phase 4 (IoT Abstractions)
    ↓
Phase 5 (BBU ApiClient migration)

Success Criteria

  • No .Abstractions project references any .Database project
  • No .Database project references non-foundational .Database (except Prism/Identity)
  • No service project references foreign .Database projects
  • All cross-service data access uses .ApiClient (Kiota HTTP)
  • CI/analyzer blocks new violations

Risks and Mitigations

Risk Mitigation
Breaking existing functionality Comprehensive test coverage before refactoring
Performance regression (HTTP vs DB) Profile critical paths, add caching, batch APIs
Migration complexity in BBU Incremental migration with feature flags
Circular dependency during transition Temporary [ADR036Exception] with expiration

Timeline Estimate

Phase Effort Dependencies
Phase 0 1-2 days None
Phase 1 1-2 days None
Phase 2 1-2 weeks Phase 1
Phase 3 3-5 days Phase 2
Phase 4 1-2 weeks Phase 2.3
Phase 5 3-4 weeks Phases 3, 4
Total 6-10 weeks

Note: Phase 3 reduced from 2-3 weeks to 3-5 days by leveraging the Passport pattern instead of creating shadow entities for every cross-component reference.